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Abstract 

Statistics are often used to reveal significant differences between online and campus-based 
education. The existence of online courses with low completion rates is often used to justify the 
inherent inferiority of online education compared to traditional classroom teaching. Our study 
revealed that this type of conclusion has little substance. We have performed three closely linked 
analyses of empirical data from Linnaeus University aimed at reaching a better understanding of 
completion rates. Differences in completion rates revealed themselves to be more substantial 
between faculties than between distribution forms. The key-factor lies in design. Courses with the 
highest completion rates had three things in common; active discussion forums, complementing 
media and collaborative activities. We believe that the time has come to move away from 
theoretical models of learning where web-based learning/distance learning/e-learning are seen as 
simply emphasizing the separation of teacher and students. Low completion rates should instead 
be addressed as a lack of insight and respect for the consequences of online pedagogical practice 
and its prerequisites. 

Keywords: completion rates, course design, online courses, web-based education. 


The muddy waters of completion 

Completion rates have been both a blindfold and a crutch in the move towards the collaborative 
and open-accessed higher education needed for the twenty-first century. In this article we argue 
that it is time to move away from a simplistic discussion centred on completion rates since they 
prevent us from addressing qualitative aspects of higher education regardless of distribution 
form. 

In higher education today we find ourselves having to handle more students who in turn are less 
well-prepared than traditional campus students and who study fewer hours per week during an 
extended period of years (Sonneby, 2012). This relatively new student group is radically different 
from the campus population, having to combine part-time studies with full-time employment, 
family and other commitments. In addition we have the problem of student compliance with 
assigned readings which have declined dramatically (Starcher & Proffitt, 2011). Finally we lack 
pedagogically based approaches to courses and programs addressing the needs of an increasingly 
diverse student population (Bates, 2010). Could these circumstances be part of the solution rather 
than the problem? 

In this article we present a three-part study of completion rates at Linnaeus University, Sweden 
and argue for a theoretical shift when discussing different distribution forms of education based 
on results (Creelman & Reneland-Forsman, 2012). We have performed three closely linked 
analyses to identify factors associated with completion rates. Online courses with low completion 
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rates are often used to justify the inherent inferiority of online education compared to traditional 
classroom teaching only based on the distribution form. Our study revealed that this type of 
conclusion has little substance. Differences in completion rates revealed themselves to be more 
substantial between, for example, faculties than between distribution forms. 


Previous research 

In the transition to using interactive media for educational purposes two important aspects of 
addressing students’ meaning making often seem to challenge course designers: the role of social 
talk and non-linearity. 

“...de centred communication has not led, in the case of e-University, to an acceptance of jointly 
constructed knowledge but rather to an emphasis on product guarantees. The fluidity of the 
internet, which media theorists proclaim as the condition for its creativity, has in fact fostered 
a pathological dislike of non-linearity...uncertainty has more often led to an aversion of risk 
rather than creativity. (Pelletier, 2005, p. 17s) 

Dealing with “social” has also proven more of a challenge than actually handling technology. 
Pitfalls are design issues of the social element such as taking communication for granted when 
interactive media is used and acknowledging the role of social but separating social 
communication from knowledge building processes (Kreijns et al., 2003). Social communication 
holds a key-role for implementing expectations on interactivity and dialogical conditions for 
learning (Jones & Issroff, 2005; Reneland-Forsman, 2009; 2012; in press). 

Social talk is also a necessary ingredient in other important key factors traced to completion rates, 
such as creating realistic expectations and a clear structure from the outset (Marald & Westerberg, 
2006; Moore, 1993; Reneland-Forsman, 2009) as well as building a sense of community in which 
students will support and encourage each other to continue even when the going gets tough 
(Reneland-Forsman, 2009; 2012; in press). Jacqueline- Aundree Baxter’s (2012) study of 
completion rates at The UK’s Open University revealed that most students expected their online 
course to be self-study and were positively surprised when online collaboration was expected of 
them. Here the popular image of online education affects student expectations. Every student 
will bring with her/him their own preconceptions, fears, insecurities and previous academic 
experience and the success of each student depends on establishing a sense of belonging, a 
supportive environment and a can-do attitude. Students have widely varying levels of experience 
in online collaboration and some may feel inadequate at first. The teacher's role of setting the 
tone and providing reassurance to the hesitant is crucial in the early stages (cf. Reneland- 
Forsman, 2012). 

Assessment and examination forms also contribute to the completion rates. Courses where 
students earn their credits step by step through regular assessed assignments had better 
completion rates than those where credits are earned in one final written examination 
(Hogskoleverket, 2011a) 

Online students who study mostly out of pure interest rather than for credits are often reluctant 
to invest the time required for a major end-of-course examination and therefore “drop out” 
before the exam. If they can earn their credits incrementally they are far more likely to complete 
the whole course (ibid). 
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Finally there is the issue of inactive students. The application process for university courses is 
long and many students who are accepted on a single optional online course may initially register 
for the course but then decide not to start for some reason. This can result in a course of 100 
registered students actually having only 30 active students. The missing 70 never really start the 
course but since the Swedish government pays universities per registered student and a further 
sum based on course completion this has a major impact on higher education financing. At the 
end of the course there may be 25/30 students who are awarded credits but because the official 
registered number was 100 the completion rate is apparently 25 %. The Swedish National Agency 
for Higher Education (Hogskoleverket, 2011b) reported that the proportion of campus-based 
inactive students is actually higher (42 %) than the percentage inactive online students (38 %). 

For a richer account of what influences retention rates we also choose to address the notion of 
“access”. New generations of students are viewed as driving the concept of education “any time, 
any place and with anyone”, something that is much more achievable with today’s technology 
(Hogskoleverket, 2008). At the same time there is a tendency to see a lower level of study skills 
and capacity in many students entering higher education today (cf. Sonneby, 2012). Access to 
technology does not in itself indicate an open road to the future since it is still to a large extent 
the already privileged in society that are able to fully exploit the opportunities offered by 
technology for empowerment and participation in adult learning. Many voices problematize a 
“digital divide” as having deeper and more complex implications than a divide between those 
with technological access or not problematizing the sustainable development of wider 
participation (Eynon & Helsper, 2011; Eynon & Malmberg, 2011; Jenkins et al., 2009; 
Livingstone et al., 2009; Livingstone & Helsper 2007; Peter & Valkenburg, 2006; Thomas, 2009). 
ICT risks at best increasing levels of participation within social groups that were already learning 
anyway (Selwyn, Gorard & Furlong, 2006). Several studies identify different aspects of digital 
literacy and an emerging digital differentiation based on level of knowledge rather than an access 
to technology. Non-participation or failures in adult and lifelong learning are deeply entrenched 
in ‘trajectories’ based on class, gender, generation, ethnicity and geography, which are established 
at an early age (Selwyn, 2009; Tuckett & Aldridge, 2009). 


Survey of completion rates 

The first study was a comparison of completion rates between different distribution forms. We 
examined the completion rates of all courses at the university for 2010 (both degree program 
courses and optional courses) using a variety of distribution forms (campus-based, online and 
blended learning with support from local learning centres). The results showed a wide range of 
completion rates with online courses having completion rates of 25-90 %. A major difficulty in 
using official completion rates is the fact that they simply measure throughput on a calendar year 
basis. Most courses in the autumn term are not formally completed until mid January and the 
credits are not registered until later on. Due to this the completion rates for autumn courses may 
appear to be very low whereas spring figures can often exceed 100 %. We concluded that these 
annual completion rates were not a reliable measurement of the success or failure of a course. 

The overall tendencies we found in our study were as follows: 

• There was no clear correlation between low completion rates and online education. There 
was similarly wide variation in completion rates between faculties and within campus- 
based courses. 

• Two faculties had higher completion rates for their online degree programmes than their 
campus-based programmes. 

European Journal of Open, Distance and e-Learning - Vol. 16 / No. 2 42 

ISSN 1027-5207 
© 2013 EDEN 



Completion Rates - A False Trail to Measuring Course Quality? 

Let's Cali in the HEROEs Instead 

Alastair Creelman, Linda Reneland-Forsman 

• Optional online courses showed significantly lower completion rates in comparison to 
their campus-based equivalents. Courses that were part of a degree programme had 
higher completion rates in both campus and online formats. 

Although there were many examples of online courses with a low completion rate there were also 
many highly successful examples with completion rates over 90 %, in some cases higher than the 
equivalent campus course. Courses that offered professional development in fields such as 
teacher training, health care or special needs education where successful completion had a direct 
impact on the students’ career development generally had much higher completion rates 
compared to courses that could be seen as simply “good to know” (e.g. basic courses in project 
management, psychology). 

Interviews with teachers and administrators showed that this was a common situation and a clear 
weakness in the system. Related to this is the fact that many online students take optional courses 
simply out of interest in the subject and are not necessarily interested in gaining credits. These 
students may seem to have failed to complete the course but are perfectly satisfied with the new 
knowledge they have gained. This type of course can be seen as contributing to lifelong learning 
rather than as part of a formal education. The Swedish government has recently proposed to 
tighten up this loophole by insisting that universities de-register inactive students after 3 weeks of 
the course and that these students will not be counted in the final completion rate statistics. 

Another significant factor in this analysis is the convergence of campus and online learning. The 
use of technology in education (VLE, recorded lectures, e-mail, instant messaging etc) has 
become ubiquitous and is equally important on campus as it is in distance education. Many full¬ 
time students take extra online courses in order to increase the credits they can earn for the term 
and we could see that a significant number of students on the universities online courses had 
addresses in the two campus towns. However many campus students overestimate their work 
capacity and discover after a few weeks that the online course is simply too demanding. As a 
result they drop out of the course. Is this a problem with online learning or a campus problem? 


Survey of inactive students 

The second analysis was a survey of this important category of registered but non-active students. 
We sent a short web-based questionnaire to 450 inactive “drop-out” students from five different 
departments at the university (from all faculties). The aim of the survey was to find out the 
reason for dropping the course and whether it was due to insufficient information or problems 
with university administration or technology. The survey completion rate was 31 % and the 
results showed that the most common reasons for not starting the course were: 

• Difficulty combining course with work commitments 

• Chose another course instead 

• Family situation (illness, bereavement, stress) 

• Lack of time 

Over 80 % were pleased with the information they had received from the university and the 
admissions process and a similar number had not experienced any significant technical 
difficulties. 

This further supports the notion that comparisons between online and campus courses are 
problematic. Students on online courses belong to a quite different target group to those on 
campus-based courses. Online students are generally over 25, already have a university degree, 
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have families and full-time employment and are not dependent on loans or other subsidies to 
finance their studies (Hogskoleverket, 2008). They tend to view online courses as lifelong 
learning and are not as highly motivated to gain credits as campus students who depend on 
successful course completion for the financing of their future studies. Few universities have fully 
grasped this vital difference and have developed specific strategies for their online prospectus. 


Course design analysis 

Thirdly a course design analysis was carried out on 12 courses (both degree programme courses 
and optional stand-alone courses). 

The pedagogic design analysis was inspired by the Theory of Transactional Distance (Moore, 
1993) since this model is useful to challenge an instmmental approach to technology and instead 
focus on the conditions for learning associated with different outcomes of course design and 
didactical considerations. The Theory of Transactional Distance describes pedagogical distance as 
a relation between structure, autonomy and communication. Courses with low interaction depend 
on high autonomy of students whereas courses with intense communication through 
motivational and structural communicative actions can compensate for lack of structure or 
autonomy. A further influence on the analysis has been studies on meaning making and the role 
of social talk for knowledge building (see Krejins et al., 2003; Reneland-Forsman, 2009; 2012). 

Guiding the analysis were levels/degrees/variations in: 

• level of interaction 

• flexibility and change in web-based interface 

• modalities used 

• evidence of collaboration 

• evidence of synchronous meetings 

• structure and lucidity 

Although based on a small material of 12 courses some tendencies were possible to observe on 
design aspects of courses. The courses with the highest completion rates had three things in 
common: active discussion forums, complementing media and collaborative activities. Courses with a higher 
completion rate than the national average (3 out of 12) all had some form of synchronous 
meeting. Alarmingly enough 10 out of 12 courses lacked stmctural pedagogic guidance — an 
orientation for students helping them to overlook the course and create meaning around it (cf. 
Marald & Westerberg, 2006; Moore, 1993). 

An alarming issue arising from our study was the significant lack of represented structure and 
orientation guiding students through a course. There are different ways of addressing the 
question of making visible a course structure and epistemology. This could be done using visual 
clues in the interface, such as a study guide or using a question approach as examples. The 
function however is crucial (cf. Marald & Westerberg, 2006; Moore, 1993). This is often supplied 
by study guides but could also be offered in the internet interface. Designing for individual tracks 
makes it more difficult to use a community for structure and guidance. Students might be 
addressing diverse aspects of a course. If this is the case, interaction rates often drop and students 
become more dependent on self-autonomy with a well-presented structure keeping them active 
and on track. The same problems probably occur in traditional distribution forms but are often 
resolved in physical encounters and activities which provide orientation for students. Programs 
with higher completion rates than traditional distribution forms all had a long tradition of 
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providing students with structured guidance in good time and a vocational orientation (based on 
the overall completion rate analysis, see also SCB 2012). 

Courses and programs had a strong linear text-orientation. Few other representational modes 
were used for students’ active content processing. Courses therefore lacked variations in terms of 
resources that could be used for jointly constructed meaning (cf. Pelletier, 2005). With few 
exceptions PDF files were piled in virtual learning environments. When film was used these were 
filmed lectures that can be viewed over again. Summing up, analysis confirms known pitfalls 
(Jones & Isroff, 2007; Krejins et al., 2003, Pelletier, 2005). The use of technology reinforces a 
traditional content distribution model of teaching rather than supporting the students’ learning 
process. A recent Norwegian report on the use of IT in higher education indicates that 
technology is mostly used to support a traditional content delivery mode rather than developing 
collaboration and student-driven learning (Norgesuniversitet, 2011; see also Pelletier, 2005). 

There is a widening digital gap between students’ use of social media to collaborate and share and 
universities’ inability to exploit the communicative possibilities of social media. 


Some reflections 

Our study of online education at Linnaeus University questioned the validity of judging a course’s 
success on completion rates since it was unclear what was actually being measured. Results 
suggest that crucial factors behind student retention are the presence of meaningful interaction 
(via face-to-face or virtual meeting spaces), clear timely information about course requirements, 
methodology and expectations and a clear link between technology and pedagogics. The study 
made the following proposals: 

• A quality assurance system adapted to the demands of online learning should be 
implemented across the whole university. Firstly based on self-assessment with the option 
of certification later 

• To create a university strategy for integrating technology in all educational activities 

• To clearly define the target group for online courses and adapt administration and 
marketing to that group. 

• Competence development and support for all teachers in IT and learning. 

• Clear routines for the deregistration of inactive students after 3 weeks of a course. 

• Better information and study guides for all online students. 

The consumption and distribution of education 

Despite the vast opportunities that social media offer to foster collaboration, sharing, creation 
and adaptation, the formal education sector in most countries remains trapped in the traditional 
classroom paradigm with the teacher/school as the provider of content and the students as 
consumers. Despite the affordances of technology, courses are mostly based on linear 
progression and content delivery and examination therefore focuses on the ability to recall and 
stmcture facts. Credits have become the hard currency of education and students are naturally 
focused on learning enough to earn credits rather than acquiring the skills they need for the 
future. 

Such a system creates a state of learned helplessness where learners expect to be consumers of 
knowledge delivered by schools and colleges. Students in the traditional model can be seen as 
charter holiday tourists, consumers of a product, where courses, learning outcomes, methods, 
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course literature and examination forms are pre-determined. At the same time, industry 
representatives are increasingly calling for backpackers: students who have learned how to find 
and filter information for themselves as well as being able to network, collaborate and be creative 
in virtual communities. 

Technology has so far mostly been used to reinforce traditional classroom practices as shown by 
the popularity of lecture capture, learning management systems (virtual classrooms) and 
smartboards. The current wave of content-based MOOCs (Massive Open Online Courses) 
offered by consortia of high status universities (Coursera, edX, Udacity, etc.) may seem 
innovative but are still heavily based on recorded lectures and traditional course structure and 
difficult to compete with for less prosperous institutions. 


A time for HEROEs 

Many of the problems discussed here can be attributed to a lack of insight and respect for the 
consequences of online pedagogical practice and its prerequisites. Up till now the main 
applications of technology in education have been to duplicate existing practices (recorded 
lectures, virtual classrooms, smartboards) or provide a reasonable substitute to the physical 
environment which is seen as default. There are too many misconceptions about the conditions 
for communication and meaning making among humans ascribed to the room or the physical 
place itself. When it comes to the digital arena we seem unable to fully think through the 
consequences of our choice of digital space. The net offers a wide range of new opportunities for 
collaboration, creation and analysis that are not being sufficiently exploited in higher education 
today. This problem can be reduced by professional development initiatives based on 
development and exchange of experience between staff members and the acknowledgement of 
changed conditions for learning. 

It is time to move away from theoretical models of learning where web-based learning/distance 
learning/e-learning are seen as simply emphasizing the separation of teacher and students. This 
results in the notion of campus education as a “shared environment” and norm whereas digital 
media are simply a means of communicating over geographical distances. This line of reasoning 
carries underlying assumptions on the normative character of physical environments as fully 
shared conditions for meaning making and human communication. 

To address the challenges of the 21st century and a more diverse student population, with the 
implied consequences for groups that lack dispositions for self-empowerment in the new learning 
cultures described here, we need informed educators that can move from campus education as 
the unquestioned norm towards students’ meaning making processes and act upon different 
affordances for meaning making and learning in particular contexts (Figure 1A & IB). 
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Figure 1A. The move towards the norm Figure IB. The move towards meaning 


European Journal of Open, Distance and e-Learning - Vol. 16 / No. 2 
ISSN 1027-5207 
© 2013 EDEN 


46 





Completion Rates - A False Trail to Measuring Course Quality? 

Let's Cali in the HEROEs Instead 

Alastair Creelman, Linda Reneland-Forsman 

Let’s call in the HEROEs (Highly Empowered Resourceful Online Educators) which means once 
and for all abandoning a consumerist approach to education applying a meaning-oriented 
approach. Acknowledging the design effects of learning environments means using a variety of 
means to trigger students’ cognitive resources. The focus on students’ active social and 
knowledge building processes regardless of distribution form also highlight the use of digital 
media in higher education as opportunities to provide experiences and orientation in a course or 
program. HEROES would analyze the conditions for knowledge building processes regardless of 
pedagogical practice. HEROEs would invite students to drag material into a learning 
environment, thus opening up the oyster — making students co-creators. As Wiley (2007) concurs, 
for the educator much of the learning, both about the subject and how to teach it, comes from 
the process of creating the object. Co-creation of knowledge using course wikis or by students 
collaborating around the creation of user-generated content are examples of such processes. Thus 
we move from the linear, content-based course to a fundamentally different model; the creation 
of a learning arena where assessment is based on successful completion of projects and where 
networking and dialogue are essential success factors. In such a connectivist environment the 
traditional learning hierarchy is evened out and students take more responsibility for their 
learning. 

The use of completion rates to measure the relative success of a course builds on a number of 
assumptions based on the traditional campus-based model for higher education. It assumes that 
students are studying for academic credentials and that this process also has financial motivation. 
As soon as net-based courses are aimed at a wider audience (lifelong learners, professional 
development, general interest) for whom the rewards of university credits are less appealing than 
the learning of new skills, the concept of completing a course becomes less relevant. The low 
completion rates of many online courses and in particular today’s mainstream MOOCs is not a 
credible gauge of course quality. We need to move the focus from a simplistic head count to 
developing strategies for increased interaction, collaboration and collective responsibility in 
online learning — HEROES could do the trick. 
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